Temporal clustering of non-stationary data

ABSTRACT

Techniques for clustering non-stationary data are disclosed. In embodiments, a method is disclosed comprising initializing a plurality of functional centroids; partitioning a non-stationary data set, using the functional centroids, into partitions, the number of partitions being equal to the number of functional centroids; generating a set of fitted functional centroids for each of the partitions; replacing at least one of the functional centroids with a corresponding fitted functional centroid if a computed energy of the corresponding fitted functional centroid is less than an energy of the at least one functional centroid; computing a summation of the energies associated with each of the functional centroids; and outputting the functional centroids upon determining that a termination condition is met.

COPYRIGHT NOTICE

This application includes material that may be subject to copyrightprotection. The copyright owner has no objection to the facsimilereproduction by anyone of the patent disclosure, as it appears in thePatent and Trademark Office files or records, but otherwise reserves allcopyright rights whatsoever.

BACKGROUND

The disclosed embodiments relate to the field of unsupervised learningand, in particular, to methods and devices for more accuratelyclustering non-stationary data.

Clustering of data is commonly used in data science and otherapplications to discover groupings of data that are otherwise unlabeled.The objective of clustering is to describe large and diverse data as anoutput of a small number of stochastic sources. When successful,clustering provides important insights into the nature of the process.For instance, clustering can show that correlations which seem to occurin a data set are actually a juxtaposition of two or more differentcorrelations—a phenomenon known as “Simpson's paradox” in behavioraldata analysis.

There are currently numerous techniques for analyzing stationary data,such as k-means clustering. However, in many applications time plays animportant part in clustering if the process is measurement over aperiod. Stream mining algorithms treat time as a computationalrestriction which forces online processing of the data. In contrast,spatiotemporal clustering such as k-means clustering focus on themovement of objects in space and therefore treat time as a privilegeddimension of the data.

Various clustering approaches still assume the sources of the data arestationary. That is, each source produces the same sequence over andover with differences due only to noise and the timing of specificsample relative to the beginning of the measurement period. Someapproaches utilize a temporal model such as Kalman filter orautoregressive-moving-average (ARMA) model that projects temporal datato a measurement space. However, these approaches assume a singlestochastic source.

The approaches above are unable to efficiently and accurately clustercomplex non-stationary data that appears in many modern systems.

BRIEF SUMMARY

The disclosed embodiments remedy these and other technical deficienciesby providing a function clustering technique that is capable of quicklyand accurately identifying one or more functional centroids for asupplied non-stationary data set.

The disclosed embodiments describe the clustering of temporal,non-stationary, data. Rather than clustering data around points in themeasurement space as employed by existing approaches, the disclosedembodiments cluster a non-stationary data set around functions from thetime to the measurement space. The identified functions can describe alinear or a polynomial trend, or they can be cyclic. Additionally, thedisclosed embodiments describe techniques for predicting the futuredistribution of the data. While traditional (stationary) clusteringalgorithms are mostly judged by their descriptive power, the disclosedembodiments temporal clustering improves upon these techniques viapredictive properties.

To accomplish these and other goals, the claims describe methods forclustering a non-stationary data set. One disclosed method comprisesinitializing a plurality of functional centroids. A non-stationary dataset is then partitioned using the functional centroids into partitionswherein the number of partitions equal to the number of functionalcentroids. The method then generates a set of fitted functionalcentroids for each of the partitions and replaces at least one of thefunctional centroids with a corresponding fitted functional centroid ifa computed energy of the corresponding fitted functional centroid isless than an energy of the at least one functional centroid. Finally,the method computes a summation of the energies associated with each ofthe functional centroids and outputs the functional centroids upondetermining that a termination condition is met. Computer-readable mediaand apparatuses implementing these and other methods are additionallydisclosed in more detail herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flow diagram illustrating a method for generating functionalcentroids according to some embodiments of the disclosure.

FIG. 2 is a flow diagram illustrating a method of partitioning datapoints between functional centroids according to some embodiments of thedisclosure.

FIG. 3 is a flow diagram illustrating a method for fitting new centroidsfor a partitioned data set according to some embodiments of thedisclosure.

FIG. 4 is a graph illustrating a non-stationary data set and multiplefunctional centroids generating using the methods described hereinaccording to some embodiments of the disclosure.

FIG. 5A is a graph illustrating a non-stationary data set and multiplefunctional centroids generating using the methods described hereinaccording to some embodiments of the disclosure.

FIG. 5B is a graph illustrating a non-stationary data set and multiplenon-functional centroids generating using pre-existing methods.

FIG. 5C is a graph illustrating the energy of the solutions computed bytraditional k-means clustering and by the disclosed embodiments invarious tests.

FIG. 5D is a diagram illustrating multiple video frames according tosome embodiments of the disclosure.

FIG. 6 is a block diagram illustrating a clustering system according tosome embodiments of the disclosure.

DETAILED DESCRIPTION

FIG. 1 is a flow diagram illustrating a method for generating functionalcentroids according to some embodiments of the disclosure.

In step 102, the method initializes one or more functional centroids.

In one embodiment, a functional centroid comprises a function taking atimestamp (or other time representation) as an input and outputting afeature vector. The number of functional centroids initialized in step102 can be adjusted based on the underlying data set the method (100) isapplied to. Various approaches to initializing a set of functionalcentroids are described herein. However, other approaches may be takenbased on the type of data being analyzed.

In each of the examples described, an input non-stationary data set ispartitioned into a series of partitions. For instance, the method canpartition the non-stationary data set into multiple time segments. Theduration of these time segments can be set according to the needs of thedata set or the processing capabilities of the computer.

In a first embodiment, the method selects a random point in each of thepartitions. Next, a function is fit using the randomly selected points.This process can be repeated multiple times to generate multiplefunctional centroids. This process is illustrated in the followingpseudo code:

TABLE 1 Given T,  

, k C ← {{c₁}, ... {c_(k)}: c_(i)~U[T] F ← {f₁, ... f_(k): f_(i) =Fit({c_(i)},  

) return Fwhere T represents the timestamped data set,

represents a function space, k represents the number of centroidsdesired, C represents a set of randomly selected points (c_(i) . . .c_(k)), U[T] represents the partitioned timestamped data set (T), Fitrepresents a fitting function, ƒ_(t) . . . ƒ_(k) represent functions infunction space

, and F represents a list of k functional centroids.

In a second embodiment, multiple points are selected for each partitionas illustrated in the following pseudo code:

TABLE 2 Given T,  

, k, d C ← {C₁ ... C_(k): C_(i) = {c_(i) ¹, ... c_(i) ^(d)} ∧ c_(i) ^(j)~ U[T] F ← {f₁, ... f_(k): f_(i) = Fit(C_(i),  

) return Fwhere Ci represents a set of randomly chosen points (one per partition),d represents the number of partitions, and j represents a givenpartition between 1 and d (remaining variables are described in thedescription of Table 1.

In a third embodiment, only one point is selected in each partition,however k-init++ initialization is used to select those points insteadof uniform selection.

In step 103, the method partitions data points between the functioncentroids identified in step 101. Further discussion of the operation ofstep 103 is provided in FIG. 2, the disclosure of which is not repeatedherein.

In general, the method iterates through each data point in the inputnon-stationary data set. For each point, the method calculates thedistance between the point and the output of each of the functionalcentroids: s−ƒ_(i)(t), where s is the value of the non-stationary dataset point at time t and ƒ_(i) is functional centroid i. The method thenselects the minimum value of |s−ƒ_(i)(t)| for each of the functionalcentroid and assigns the point (s, t) to a centroid set (T_(c)) for thatcentroid c.

In step 105, the method then fits new centroids for each partitiongenerated in step 103. Further discussion of the operation of step 105is provided in FIG. 3, the disclosure of which is not repeated herein.

In general, the method selects each centroid set and fits a functionbased on the data points in that set. The method then determines if theenergy of the fitted function is lower than the energy of the currentcentroid function. If so, the centroid function is set to the fittedfunction and, if not, the fitted function is discarded. The energy ofthe chosen function is then added to a set of energies for each centroidfunction. The “energy” of a centroid function is described more fully inthe description of step 107.

In step 107, the method sums the centroid energies stored in step 105.As described in step 105, the energy of a function can be represented asfollows:

$\begin{matrix}{{E_{f}(s)} = {\sum\limits_{{\langle{s,t}\rangle} \Subset S}{{s - {f(t)}}}}} & {{EQUATION}\mspace{20mu} 1}\end{matrix}$

where

s, t

is a data point (s) at time (t) within a non-stationary data set (S) andƒ is a centroid function. As illustrated in Equation 1, an idealcentroid function would result in an energy Eƒ(s) of zero. In thatscenario, the centroid function perfectly describes all points exactly.In some embodiments, the value of |s−(t)| may be weighted to account foroutlying data points. For example, if n−1 points of n data points matchexactly and one data point is significantly off, the method may weighthe data points to avoid such an outlier from skewing the energy of thecentroid function.

In step 109, the method determines if the summed energies calculated instep 107 result in a net energy decrease. If so, the method continues toexecute steps 103, 105, 107, and 109 until those steps no longer resultin an energy decrease. In alternative embodiments, the method mayexecute steps 103, 105, 107, and 109 a preset number of times beforeterminating (to avoid infinite loops).

In step 111, the method outputs the final function centroids. In someembodiments, the functional centroid may comprise polynomialexpressions. In this embodiment, the method outputs coefficients andpowers of the expression. In other embodiments, the method may outputthe function itself. In either event, the functional centroids can beprovided to downstream software application for further applications,some of which are described herein.

FIG. 2 is a flow diagram illustrating a method of partitioning datapoints between functional centroids according to some embodiments of thedisclosure.

In step 201, the method selects a non-stationary data point.

In the illustrated embodiment, a non-stationary data set comprises a setof points

s, t

where t represents a time and s represents a corresponding value at timet. One example of a non-stationary data set (depicted in FIG. 4) isdwell time of a web application over time. In this example, the value ofs corresponds to a measure of time (e.g., 3 seconds) while the value oft corresponds to a timestamp that the dwell time is collected. Each datapoint in the non-stationary data set independently measures a singleinteraction of one of a large number of users with a single emailmessage. The non-stationary data can be sampled over consecutive days(e.g., seven days as illustrated in FIG. 4). As can be seen in FIG. 4,the range of measured dwell time starts at one millisecond and goes allthe way to e^(17.5) milliseconds, or about 11 hours. A very short dwelltime is typical of inbox cleanup, longer dwell time indicates readingthe message and, sometimes, replying to it. Dwell time might be evenlonger if the user interacts with other web sites, or with humanservices, while reading the message (e.g., if the user synchronizes ahotel while looking at a trip itinerary). The longest dwell times may beusers neglecting to close the browser window containing the email (i.e.,they are outliers). Analyzing the data with temporal-clustering, as willbe explained herein, reveals three different temporal behaviors (401,403, 405) that can be represented as functional centroids. All threefollow a daily cycle. The strongest cycle is that of short dwell time(405) with a mean between twenty and five hundred milliseconds. If weaccept that this kind of activity indicates inbox cleaning then we learnthat this activity is more dominant around 2 PM EST based on thefunctional centroid (405) identified using the methods described herein.Such insights, if validated by other means, can lead to further productideas and improve downstream software applications.

As illustrated in FIG. 4, the set of points

s, t

are not exclusive. Various values of s may exist for any given t.Continuing the example, there are potentially a large range of values ofs at any given time t. In the illustrated embodiment, each point isselected regardless of its values for s or t.

In step 203, the method finds a centroid function closest to the datapoint selected in step 201.

As described in connection with FIG. 1, during operation the method inFIG. 2 is provided with a set of functional centroids. These functionalcentroids comprise functions that take, as an input, a time and output adata point in the non-stationary data set. To find the functionalcentroid that is closest to a given point, the method in step 203evaluates each value (s₁, s₂, . . . s_(k)) for each centroid function(ƒ₁, ƒ₂, . . . ƒ_(k)). Then, for each value (s₁, s₂, . . . s_(k)), themethod computes the distance between the actual point selected in 201(s) and the results of the centroid functions (|s−s₁|, |s−s₂| . . .|s−s_(k)|). The minimum value (argmin) of this set is chosen as theclosest functional centroid. In this manner, the method identifies thecentroid function that most closely matches the partitioned data.

In step 205, the method adds the data point selected in step 201 to apartition associated with the centroid function selected in step 203.

In one embodiment, prior to executing step 201, the method initializes aset of partitions (T₁, T₂, . . . T_(C)) to null sets, where c representsa centroid function. After identifying the closest centroid function,the method adds the data point from step 201 into the set (e.g., theunion of

s, t

and T_(c)). The resulting set represents the partitioning ofnon-stationary data by centroid function.

In step 207, the method determines if all data points in thenon-stationary data set have been analyzed. If not, the method continuesto re-execute steps 201, 203, and 205 until all points have beenassigned to a partition. Once all points have been processed, the methodends.

FIG. 3 is a flow diagram illustrating a method for fitting new centroidsfor a partitioned data set according to some embodiments of thedisclosure.

In step 301, the method selects a centroid.

In the illustrated embodiment, selecting a centroid comprises selectingpartition generated in FIG. 2. That is, in step 301, the method selectsa set of non-stationary data points partitioned based on the minimumdistances to other centroid functions.

In step 303, the method fits a function using the data points in thepartition associated with the centroid.

In the illustrated embodiment, fitting a function refers to identifyinga function that most closely generates the data points used to fit thefunction. In the illustrated embodiment, various function fittingtechniques can be used. For example, in a first embodiment, the methodmay utilize polynomial function fitting to identify a function fittingthe partition data points. In some embodiments, the polynomial degreemay be three or may be adjusted as needed. As a second example, themethod may utilize single cosine fitting. In some embodiments, whenusing single cosine fitting the bias, frequency, and phase of the cosinemay be optimized prior to fitting. For example, the frequency may employL2-regularization to improve prediction of the fitting. As a thirdexample, a nonequispaced fast Fourier transform (NFFT) may be used. Inthis example, a subset of the frequencies may be selected for fitting(e.g., eight of 256 frequencies).

Although the previous example illustrates various alternatives forfunctional fitting, other techniques may be used. In general, eachtechnique requires a range of meta parameters. Additionally, given adataset and absent knowledge of the temporal pattern of the data, thefitting algorithm itself should be treated as a meta-parameter.Moreover, in periodic data, the period is a meta-parameter. Last, thenumber of centroids is a meta-parameter. In some embodiments, themeta-parameters are selected by optimizing the predictive energy of theresulting centroids. In some embodiments, the number of centroids isexcluded from this optimization since the energy of a solution decreasesad the number of centroids increases. However, the disclosed embodimentscan only improve with respect to k-means if it finds another trend whichcan be predicted from the data. Therefore, in some embodiments a numberof centroids is selected which minimizes the energy of the disclosedembodiments when compared to the energy of traditional clusteringtechniques such as k-means clustering. In some embodiments, parametertuning is conducted on a separate period of the data than the one laterused in experimentation.

In step 305, the method determines if the energy of the fitted functionis less than the energy of the previous function.

As described above, the energy of a function is computed by summing thedistances between an output value of the function and the expectedvalue. In the illustrated embodiment, the existing energy prior tofitting the function in step 303 is cached. In step 305, the methodcomputes the energy for the fitted function. In the illustratedembodiment, this step entails summing the distances between the outputof the fitted function and the known data points in the activepartition. In step 305, the method compares the two values (cached andcomputed). If the computed value is less than the cached value, theenergy has decreased and the fitted function is a better approximationof the partition; thus, the method executes step 307.

In step 307, the method has detected that the fitted function betterfits the centroid partition (due to the decrease in functional energy).In this case, the method replaces the current function for the partitionwith the fitted function. Alternatively, if the method detects that theenergy change is nil, or if the fitted function increases the energy ofthe candidate function, the method discards the fitted function.

In step 309, the method saves the centroid function energy. In someembodiments, step 309 may be optional. As described previously inconnection with FIG. 1, individual function energies are utilized(summed) to determine when the terminate the clustering procedure.

In step 311, the method determines if any centroids/partitions remain tobe processed. If so, the method repeats steps 301, 303, 305, 307, 309,and 311 for each remaining partition/centroid. If not, the method ends.

FIG. 5A is a graph illustrating a non-stationary data set and multiplefunctional centroids generating using the methods described hereinaccording to some embodiments of the disclosure. FIG. 5B is a graphillustrating a non-stationary data set and multiple non-functionalcentroids generating using pre-existing methods.

As illustrated, a non-stationary data set representing weather data isgraphically displayed as graphs 501 and 511. In both FIGS. 5A and 5B,the weather data is plotted as temperature given a day and theunderlying data is the same. In both graphs, the data is segmented intoa training period and prediction period at point (509, 519). That is,data to the left of point (509, 519) comprises training data and data tothe right of point (509, 519) comprises prediction data.

In the illustrated embodiment, the first graph 501 illustrates threefunctional centroids (503, 505, 507) using the methods disclosed herein.The second graph 511 illustrates three k-means centroids (513, 515, 517)generated using traditional k-means clustering.

In the illustrated embodiment, the energy of the data in the time unitsappearing after point (509, 519) are measured with respect to thefunctional centroids (503, 507, 509) and the k-means centroids (513,517, 519). In the illustrated embodiment, the energy for the functioncentroid clusters (FIG. 5A) is approximately 51 million and that of thetraditional k-means centroids (FIG. 5B) is about 62 million. Hence, theimprovement of function clustering is approximately 51/62, or 82%. Inother words, th disclosed embodiments are able to explain 18% of theenergy of future points by the temporal trends in the data when comparedwith existing clustering techniques such as k-means clustering.

FIG. 5C is a graph illustrating the energy of the solutions computed bytraditional k-means clustering and by the disclosed embodiments invarious tests. In the illustrated embodiment, the claimed clusteringmethod (FunKMeans) is compared with traditional k-means (KMeans). Theillustrated graph 520 illustrates the result when comparing thealgorithms over multiple windows of training data (x-axis) and plots theenergy (y-axis for each window). As can be seen, while the performanceof the disclosed clustering methods varies in this experiment, it isconsistently better than that of traditional k-means.

Additionally, the disclosed embodiments were test on various datasets:

TABLE 3 Name Instances Features Distinct Timestamps Weather 140,000 32,555 Smoking 876 4 16 Synthetic 200,000 1 100,000 Search 250,000 1237,000 NJ Transit 1,000,000 1 150,00

In these datasets, Weather contains the daily summary of the minimum andmaximum temperatures and the precipitation in thousands of groundstations provided by the U.S. Historical Climatology Network (USHCN).The Smoking data set contains four categories of smoking habits overseveral years (controlling for the state variable) provided by theCenters for Disease Control and Prevention (CDC). The NJ Transit dataset comprises delay information from the New Jersey transit datasetavailable from Kaggle. Additionally, two more data sets were added.First, a Synthetic data set comprising two cosines

$3 + {\sin\;\left( {{4\Pi\mspace{11mu} t} + \frac{\Pi}{4}} \right)\mspace{14mu}{and}\mspace{14mu} 5\mspace{14mu}{\sin\left( {{10\Pi\mspace{11mu} t} + \frac{\Pi}{4}} \right)}}$

with white noise—M[0,1] added to each sample. Second, a Search data setcomprising metrics related to a search engine results page (e.g., clickthrough data)

For each of the above data sets, a large number of observations fromdifferent sliding windows (e.g., points 509, 519) were collected on eachdata set and the energy of the prediction horizon (e.g., right of points509, 519) was calculated. The Wilcoxon signed-rank test to reject thehypothesis that the energy of the solution computed by k-means is equalto that computed by the disclosed methods. The results of the testing isprovided in Table 3 below:

TABLE 4 Fun-k k-means p- Best Name τ ℏ avg (stdev) avg (stdev) value fitk Weather 2y 2y 3.76E+8 4.78E+8 0.0  DFT 3  (5.1E+6)  (5.4E+6) Smoking8y 2y 6.3 (0.21) 8.0 (0.35) 0.015 Polyfit 2 Synthetic 2t 1t 1.657E+3(55)   4.79E+3 (101) 0.0  DFT 2 Search 1m 1w 2.435E+4 (537) 2.438E+4(539) 0.001 DFT 3 NJ 3w 1w 8.38E+5 8.62E+5 0.07  Sine 2 Transit(3.56E+5) (3.76E+5)

In the illustrated table, T represents the test window and h representthe prediction window (left and right of 509, 519, respectively). Thebest fit column indicates the chosen curve fitting algorithm used and krepresents the number of centroids. Thus, as illustrated the disclosedmethods (Fun-k means) significantly improves clustering in the utilizeddatasets.

While predictive power is the most important indication that aclustering result is valuable, the main goal of clustering is often toproduce insights rather than predict the future. To exemplify thedescriptive power of temporal clustering, the disclosed methods wereapplied to video data. Video can be considered as a three-dimensionaldata set: X, Y, and grayscale value. In the embodiment illustrated inFIG. 5D, 250 consecutive frames were samples. Both traditional k-meansand function centroid approaches were applied to the samples. Onaverage, the energy of the clustering found by the disclosed embodimentsmeans was 96% of the energy of k-means clustering.

FIG. 5D depicts a sample of the result of the clustering: on the toprow, four frames (522 a-522 d) are shown from the original video data.The frames (526 a-526 d) of the bottom raw are the results of k-meansclustering. As can be seen, k-means clusters together areas with roughlythe same color and takes into account spatial proximity. For instance,the left and right black rails are assigned to different clusters. Butthe smoke and the smokestack, which are both black, are assigned to thesame cluster as the right rail.

The frames (524 a-524 d) middle row depicts clustering using thedisclosed embodiments. In contrast to frames (526 a-526 d) theclustering using the disclosed embodiments identifies moving parts ofthe frames: the smoke and the upper part of the smokestack are assignedto their own cluster (the upper side of the right rail is assigned tothat cluster as well, based on proximity). Hence, the disclosedembodiments are capable of identifying the temporal feature of the videodata.

FIG. 6 is a block diagram illustrating a clustering system according tosome embodiments of the disclosure.

System 600 may include many more or less components than those shown inFIG. 6. However, the components shown are sufficient to disclose anillustrative embodiment for implementing the methods describedpreviously.

As shown in FIG. 6, computing system 600 includes a processing unit(CPU) 622 in communication with a mass memory 630 via a bus 624.Computing system 600 also includes a power supply 626, one or morenetwork interfaces 650, an audio interface 652, a display 654, a keypad656, an illuminator 658, an input/output interface 660, and a camera(s)or other optical, thermal or electromagnetic sensors 662. Computingsystem 600 can include one camera/sensor 662, or a plurality ofcameras/sensors 662, as understood by those of skill in the art.

Power supply 626 provides power to computing system 600. A rechargeableor non-rechargeable battery may be used to provide power. The power mayalso be provided by an external power source, such as an AC adapter or apowered docking cradle that supplements and/or recharges a battery.

Computing system 600 may optionally communicate with a base station (notshown), or directly with another computing device. Network interface 650includes circuitry for coupling computing system 600 to one or morenetworks, and is constructed for use with one or more communicationprotocols and technologies. Network interface 650 is sometimes known asa transceiver, transceiving device, or network interface card (NIC).

Audio interface 652 is arranged to produce and receive audio signalssuch as the sound of a human voice. For example, audio interface 652 maybe coupled to a speaker and microphone (not shown) to enabletelecommunication with others and/or generate an audio acknowledgementfor some action. Display 654 may be a liquid crystal display (LCD), gasplasma, light emitting diode (LED), or any other type of display usedwith a computing device. Display 654 may also include a touch sensitivescreen arranged to receive input from an object such as a stylus or adigit from a human hand.

Keypad 656 may comprise any input device arranged to receive input froma user. For example, keypad 656 may include a push button numeric dial,or a keyboard. Keypad 656 may also include command buttons that areassociated with selecting and sending images. Illuminator 658 mayprovide a status indication and/or provide light. Illuminator 658 mayremain active for specific periods of time or in response to events. Forexample, when illuminator 658 is active, it may backlight the buttons onkeypad 656 and stay on while the system is powered. Also, illuminator658 may backlight these buttons in various patterns when particularactions are performed, such as dialing another client device.Illuminator 658 may also cause light sources positioned within atransparent or translucent case of the client device to illuminate inresponse to actions.

Computing system 600 also comprises input/output interface 660 forcommunicating with external devices or other input or devices not shownin FIG. 6. Input/output interface 660 can utilize one or morecommunication technologies, such as USB, infrared, Bluetooth™, or thelike.

Mass memory 630 includes a RAM 632, a ROM 634, and other storage means.Mass memory 630 illustrates another example of computer storage mediafor storage of information such as computer readable instructions, datastructures, program modules or other data. Mass memory 630 stores abasic input/output system (“BIOS”) 640 for controlling low-leveloperation of computing system 600. The mass memory also stores anoperating system 641 for controlling the operation of computing system600. It will be appreciated that this component may include a generalpurpose operating system such as a version of UNIX, or LINUX™, or aspecialized client communication operating system such as WindowsClient™, or the Symbian® operating system. The operating system mayinclude, or interface with a Java virtual machine module that enablescontrol of hardware components and/or operating system operations viaJava application programs.

Memory 630 further includes one or more data stores, which can beutilized by computing system 600 to store, among other things, functionclustering application 642 and/or other data. For example, data storesmay be employed to store information that describes various capabilitiesof computing system 600. The information may then be provided to anotherdevice based on any of a variety of events, including being sent as partof a header during a communication, sent upon request, or the like. Atleast a portion of the capability information may also be stored on adisk drive or other storage medium (not shown) within computing system600. Function clustering application 642 may include computer executableinstructions which, when executed by computing system 600, enable thefunction clustering of non-stationary data in accordance with themethods described above.

The present disclosure has been described with reference to theaccompanying drawings, which form a part hereof, and which show, by wayof illustration, certain example embodiments. Subject matter may,however, be embodied in a variety of different forms and, therefore,covered or claimed subject matter is intended to be construed as notbeing limited to any example embodiments set forth herein; exampleembodiments are provided merely to be illustrative. Likewise, areasonably broad scope for claimed or covered subject matter isintended. Among other things, for example, subject matter may beembodied as methods, devices, components, or systems. Accordingly,embodiments may, for example, take the form of hardware, software,firmware or any combination thereof (other than software per se). Thefollowing detailed description is, therefore, not intended to be takenin a limiting sense.

Throughout the specification and claims, terms may have nuanced meaningssuggested or implied in context beyond an explicitly stated meaning.Likewise, the phrase “in one embodiment” as used herein does notnecessarily refer to the same embodiment and the phrase “in anotherembodiment” as used herein does not necessarily refer to a differentembodiment. It is intended, for example, that claimed subject matterinclude combinations of example embodiments in whole or in part.

In general, terminology may be understood at least in part from usage incontext. For example, terms, such as “and”, “or”, or “and/or,” as usedherein may include a variety of meanings that may depend at least inpart upon the context in which such terms are used. Typically, “or” ifused to associate a list, such as A, B or C, is intended to mean A, B,and C, here used in the inclusive sense, as well as A, B or C, here usedin the exclusive sense. In addition, the term “one or more” as usedherein, depending at least in part upon context, may be used to describeany feature, structure, or characteristic in a singular sense or may beused to describe combinations of features, structures or characteristicsin a plural sense. Similarly, terms, such as “a,” “an,” or “the,” again,may be understood to convey a singular usage or to convey a pluralusage, depending at least in part upon context. In addition, the term“based on” may be understood as not necessarily intended to convey anexclusive set of factors and may, instead, allow for existence ofadditional factors not necessarily expressly described, again, dependingat least in part on context.

The present disclosure is described below with reference to blockdiagrams and operational illustrations of methods and devices. It isunderstood that each block of the block diagrams or operationalillustrations, and combinations of blocks in the block diagrams oroperational illustrations, can be implemented by means of analog ordigital hardware and computer program instructions. These computerprogram instructions can be provided to a processor of a general purposecomputer to alter its function as detailed herein, a special purposecomputer, ASIC, or other programmable data processing apparatus, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, implement thefunctions/acts specified in the block diagrams or operational block orblocks. In some alternate implementations, the functions/acts noted inthe blocks can occur out of the order noted in the operationalillustrations. For example, two blocks shown in succession can in factbe executed substantially concurrently or the blocks can sometimes beexecuted in the reverse order, depending upon the functionality/actsinvolved.

These computer program instructions can be provided to a processor of: ageneral purpose computer to alter its function to a special purpose; aspecial purpose computer; ASIC; or other programmable digital dataprocessing apparatus, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, implement the functions/acts specified in the block diagramsor operational block or blocks, thereby transforming their functionalityin accordance with embodiments herein.

For the purposes of this disclosure a computer readable medium (orcomputer-readable storage medium/media) stores computer data, which datacan include computer program code (or computer-executable instructions)that is executable by a computer, in machine readable form. By way ofexample, and not limitation, a computer readable medium may comprisecomputer readable storage media, for tangible or fixed storage of data,or communication media for transient interpretation of code-containingsignals. Computer readable storage media, as used herein, refers tophysical or tangible storage (as opposed to signals) and includeswithout limitation volatile and non-volatile, removable andnon-removable media implemented in any method or technology for thetangible storage of information such as computer-readable instructions,data structures, program modules or other data. Computer readablestorage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM,flash memory or other solid state memory technology, CD-ROM, DVD, orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other physical ormaterial medium which can be used to tangibly store the desiredinformation or data or instructions and which can be accessed by acomputer or processor.

For the purposes of this disclosure the term “server” should beunderstood to refer to a service point which provides processing,database, and communication facilities. By way of example, and notlimitation, the term “server” can refer to a single, physical processorwith associated communications and data storage and database facilities,or it can refer to a networked or clustered complex of processors andassociated network and storage devices, as well as operating softwareand one or more database systems and application software that supportthe services provided by the server. Servers may vary widely inconfiguration or capabilities, but generally a server may include one ormore central processing units and memory. A server may also include oneor more mass storage devices, one or more power supplies, one or morewired or wireless network interfaces, one or more input/outputinterfaces, or one or more operating systems, such as Windows Server,Mac OS X, Unix, Linux, FreeBSD, or the like.

For the purposes of this disclosure a “network” should be understood torefer to a network that may couple devices so that communications may beexchanged, such as between a server and a client device or other typesof devices, including between wireless devices coupled via a wirelessnetwork, for example. A network may also include mass storage, such asnetwork attached storage (NAS), a storage area network (SAN), or otherforms of computer or machine readable media, for example. A network mayinclude the Internet, one or more local area networks (LANs), one ormore wide area networks (WANs), wire-line type connections, wirelesstype connections, cellular or any combination thereof. Likewise,sub-networks, which may employ differing architectures or may becompliant or compatible with differing protocols, may interoperatewithin a larger network. Various types of devices may, for example, bemade available to provide an interoperable capability for differingarchitectures or protocols. As one illustrative example, a router mayprovide a link between otherwise separate and independent LANs.

A communication link or channel may include, for example, analogtelephone lines, such as a twisted wire pair, a coaxial cable, full orfractional digital lines including T1, T2, T3, or T4 type lines,Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines(DSLs), wireless links including satellite links, or other communicationlinks or channels, such as may be known to those skilled in the art.Furthermore, a computing device or other related electronic devices maybe remotely coupled to a network, such as via a wired or wireless lineor link, for example.

For purposes of this disclosure, a “wireless network” should beunderstood to couple client devices with a network. A wireless networkmay employ stand-alone ad-hoc networks, mesh networks, Wireless LAN(WLAN) networks, cellular networks, or the like. A wireless network mayfurther include a system of terminals, gateways, routers, or the likecoupled by wireless radio links, or the like, which may move freely,randomly or organize themselves arbitrarily, such that network topologymay change, at times even rapidly.

A wireless network may further employ a plurality of network accesstechnologies, including Wi-Fi, Long Term Evolution (LTE), WLAN, WirelessRouter (WR) mesh, or 2nd, 3rd, or 4th generation (2G, 3G, or 4G)cellular technology, or the like. Network access technologies may enablewide area coverage for devices, such as client devices with varyingdegrees of mobility, for example.

For example, a network may enable RF or wireless type communication viaone or more network access technologies, such as Global System forMobile communication (GSM), Universal Mobile Telecommunications System(UMTS), General Packet Radio Services (GPRS), Enhanced Data GSMEnvironment (EDGE), 3GPP Long Term Evolution (LTE), LTE Advanced,Wideband Code Division Multiple Access (WCDMA), Bluetooth, 802.11b/g/n,or the like. A wireless network may include virtually any type ofwireless communication mechanism by which signals may be communicatedbetween devices, such as a client device or a computing device, betweenor within a network, or the like.

A computing device may be capable of sending or receiving signals, suchas via a wired or wireless network, or may be capable of processing orstoring signals, such as in memory as physical memory states, and may,therefore, operate as a server. Thus, devices capable of operating as aserver may include, as examples, dedicated rack-mounted servers, desktopcomputers, laptop computers, set top boxes, integrated devices combiningvarious features, such as two or more features of the foregoing devices,or the like. Servers may vary widely in configuration or capabilities,but generally a server may include one or more central processing unitsand memory. A server may also include one or more mass storage devices,one or more power supplies, one or more wired or wireless networkinterfaces, one or more input/output interfaces, or one or moreoperating systems, such as Windows Server, Mac OS X, Unix, Linux,FreeBSD, or the like.

For purposes of this disclosure, a client (or consumer or user) devicemay include a computing device capable of sending or receiving signals,such as via a wired or a wireless network. A client device may, forexample, include a desktop computer or a portable device, such as acellular telephone, a smart phone, a display pager, a radio frequency(RF) device, an infrared (IR) device an Near Field Communication (NFC)device, a Personal Digital Assistant (PDA), a handheld computer, atablet computer, a phablet, a laptop computer, a set top box, a wearablecomputer, smart watch, an integrated or distributed device combiningvarious features, such as features of the forgoing devices, or the like.

A client device may vary in terms of capabilities or features. Claimedsubject matter is intended to cover a wide range of potentialvariations. For example, a simple smart phone, phablet or tablet mayinclude a numeric keypad or a display of limited functionality, such asa monochrome liquid crystal display (LCD) for displaying text. Incontrast, however, as another example, a web-enabled client device mayinclude a high resolution screen, one or more physical or virtualkeyboards, mass storage, one or more accelerometers, one or moregyroscopes, global positioning system (GPS) or otherlocation-identifying type capability, or a display with a high degree offunctionality, such as a touch-sensitive color 2D or 3D display, forexample.

A client device may include or may execute a variety of operatingsystems, including a personal computer operating system, such as aWindows, iOS or Linux, or a mobile operating system, such as iOS,Android, or Windows Mobile, or the like.

A client device may include or may execute a variety of possibleapplications, such as a client software application enablingcommunication with other devices, such as communicating one or moremessages, such as via email, for example Yahoo!® Mail, short messageservice (SMS), or multimedia message service (MMS), for example Yahoo!Messenger®, including via a network, such as a social network,including, for example, Tumblr®, Facebook®, LinkedIn®, Twitter®,Flickr®, or Google+®, Instagram™, to provide only a few possibleexamples. A client device may also include or execute an application tocommunicate content, such as, for example, textual content, multimediacontent, or the like. A client device may also include or execute anapplication to perform a variety of possible tasks, such as browsing,searching, playing or displaying various forms of content, includinglocally stored or streamed video, or games (such as fantasy sportsleagues). The foregoing is provided to illustrate that claimed subjectmatter is intended to include a wide range of possible features orcapabilities.

For the purposes of this disclosure a module is a software, hardware, orfirmware (or combinations thereof) system, process or functionality, orcomponent thereof, that performs or facilitates the processes, features,and/or functions described herein (with or without human interaction oraugmentation). A module can include sub-modules. Software components ofa module may be stored on a computer readable medium for execution by aprocessor. Modules may be integral to one or more servers, or be loadedand executed by one or more servers. One or more modules may be groupedinto an engine or an application.

For the purposes of this disclosure the term “user”, “subscriber”“consumer” or “customer” should be understood to refer to a user of anapplication or applications as described herein and/or a consumer ofdata supplied by a data provider. By way of example, and not limitation,the term “user” or “subscriber” can refer to a person who receives dataprovided by the data or service provider over the Internet in a browsersession, or can refer to an automated software application whichreceives the data and stores or processes the data.

Those skilled in the art will recognize that the methods and systems ofthe present disclosure may be implemented in many manners and as suchare not to be limited by the foregoing exemplary embodiments andexamples. In other words, functional elements being performed by singleor multiple components, in various combinations of hardware and softwareor firmware, and individual functions, may be distributed among softwareapplications at either the client level or server level or both. In thisregard, any number of the features of the different embodimentsdescribed herein may be combined into single or multiple embodiments,and alternate embodiments having fewer than, or more than, all of thefeatures described herein are possible.

Functionality may also be, in whole or in part, distributed amongmultiple components, in manners now known or to become known. Thus,myriad software/hardware/firmware combinations are possible in achievingthe functions, features, interfaces and preferences described herein.Moreover, the scope of the present disclosure covers conventionallyknown manners for carrying out the described features and functions andinterfaces, as well as those variations and modifications that may bemade to the hardware or software or firmware components described hereinas would be understood by those skilled in the art now and hereafter.

Furthermore, the embodiments of methods presented and described asflowcharts in this disclosure are provided by way of example in order toprovide a more complete understanding of the technology. The disclosedmethods are not limited to the operations and logical flow presentedherein. Alternative embodiments are contemplated in which the order ofthe various operations is altered and in which sub-operations describedas being part of a larger operation are performed independently.

While various embodiments have been described for purposes of thisdisclosure, such embodiments should not be deemed to limit the teachingof this disclosure to those embodiments. Various changes andmodifications may be made to the elements and operations described aboveto obtain a result that remains within the scope of the systems andprocesses described in this disclosure.

1-23. (canceled)
 24. A method comprising: initializing functionalcentroids, a respective functional centroid in the functional centroidsaccepting a timestamp as an input and outputting at least one data pointbased on the timestamp; replacing at least one functional centroid inthe functional centroids with a corresponding fitted functional centroidto obtain final functional centroids if a computed energy of thecorresponding fitted functional centroid is less than an energy of theat least one functional centroid; and outputting the final functionalcentroids.
 25. The method of claim 24, further comprising: partitioninga non-stationary data set, using the functional centroids, intopartitions, a number of partitions being equal to a number of functionalcentroids; and generating a set of fitted functional centroids for eachof the partitions, the set of fitted functional centroids including thecorresponding fitted functional centroid.
 26. The method of claim 25,wherein partitioning the non-stationary data set comprises: identifying,for each point in the non-stationary data set, a closest functionalcentroid generating an output value closest to a respective point; andassigning each point to a partition based on the closest functionalcentroid.
 27. The method of claim 25, further comprising: computing asum of energies associated with each of the functional centroids; anddetermining that the sum does not result in a net energy decrease priorto outputting the final functional centroids.
 28. The method of claim27, an energy of a function computed by summing differences betweenknown points at respective times and outputs of the function forcorresponding respective times.
 29. The method of claim 27, whereindetermining that the sum of the energies does not result in a net energydecrease comprises: computing a first energy for the functionalcentroids prior to generating the set of fitted functional centroids;computing a second energy for the functional centroids after replacingat least one of the functional centroids with a corresponding fittedfunctional centroid; and determining that the sum of the energies doesnot result in a net energy decrease if the second energy is not lessthan the first energy.
 30. The method of claim 27, further comprisingperforming a second partitioning of the non-stationary data set usingthe functional centroids upon determining that the sum represents a netenergy decrease.
 31. The method of claim 25, wherein generating the setof fitted functional centroids comprises: fitting a function based ondata points associated with a partition to generate a fitted function;determining if an energy of the fitted function is less than an energyof a corresponding functional centroid; and using the fitted function asthe corresponding functional centroid.
 32. A non-transitory computerreadable storage medium for tangibly storing computer programinstructions capable of being executed by a computer processor, thecomputer program instructions defining steps of: initializing functionalcentroids, a respective functional centroid in the functional centroidsaccepting a timestamp as an input and outputting at least one data pointbased on the timestamp; replacing at least one functional centroid inthe functional centroids with a corresponding fitted functional centroidto obtain final functional centroids if a computed energy of thecorresponding fitted functional centroid is less than an energy of theat least one functional centroid; and outputting the final functionalcentroids.
 33. The non-transitory computer readable storage medium ofclaim 32, further comprising: partitioning a non-stationary data set,using the functional centroids, into partitions, a number of partitionsbeing equal to a number of functional centroids; and generating a set offitted functional centroids for each of the partitions, the set offitted functional centroids including the corresponding fittedfunctional centroid.
 34. The non-transitory computer readable storagemedium of claim 33, wherein partitioning the non-stationary data setcomprises: identifying, for each point in the non-stationary data set, aclosest functional centroid generating an output value closest to arespective point; and assigning each point to a partition based on theclosest functional centroid.
 35. The non-transitory computer readablestorage medium of claim 33, further comprising: computing a sum ofenergies associated with each of the functional centroids; anddetermining that the sum does not result in a net energy decrease priorto outputting the final functional centroids.
 36. The non-transitorycomputer readable storage medium of claim 35, an energy of a functioncomputed by summing differences between known points at respective timesand outputs of the function for corresponding respective times.
 37. Thenon-transitory computer readable storage medium of claim 35, whereindetermining that the sum of the energies does not result in a net energydecrease comprises: computing a first energy for the functionalcentroids prior to generating the set of fitted functional centroids;computing a second energy for the functional centroids after replacingat least one of the functional centroids with a corresponding fittedfunctional centroid; and determining that the sum of the energies doesnot result in a net energy decrease if the second energy is not lessthan the first energy.
 38. The non-transitory computer readable storagemedium of claim 34, wherein generating the set of fitted functionalcentroids comprises: fitting a function based on data points associatedwith a partition to generate a fitted function; determining if an energyof the fitted function is less than an energy of a correspondingfunctional centroid; and using the fitted function as the correspondingfunctional centroid.
 39. A device comprising: a processor configured to:initialize functional centroids, a respective functional centroid in thefunctional centroids accepting a timestamp as an input and outputting atleast one data point based on the timestamp; replace at least onefunctional centroid in the functional centroids with a correspondingfitted functional centroid to obtain final functional centroids if acomputed energy of the corresponding fitted functional centroid is lessthan an energy of the at least one functional centroid; and output thefinal functional centroids.
 40. The device of claim 39, the processorfurther configured to: partition a non-stationary data set, using thefunctional centroids, into partitions, a number of partitions beingequal to a number of functional centroids; and generate a set of fittedfunctional centroids for each of the partitions, the set of fittedfunctional centroids including the corresponding fitted functionalcentroid.
 41. The device of claim 40, wherein partitioning thenon-stationary data set comprises: identifying, for each point in thenon-stationary data set, a closest functional centroid generating anoutput value closest to a respective point; and assigning each point toa partition based on the closest functional centroid.
 42. The device ofclaim 40, the processor further configure to: compute a sum of energiesassociated with each of the functional centroids; and determine that thesum does not result in a net energy decrease prior to outputting thefinal functional centroids.
 43. The device of claim 40, whereingenerating the set of fitted functional centroids comprises: fitting afunction based on data points associated with a partition to generate afitted function; determining if an energy of the fitted function is lessthan an energy of a corresponding functional centroid; and using thefitted function as the corresponding functional centroid.